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AN  EVALUATION  OF  CONVENTIONAL  CORRELATION  METHODS  WHEN 
MATCHING  INFRARED  IMAGERY  TO  PANCHROMATIC  IMAGERY 


INTRODUCTION 

One  of  the  basic  objectives  in  image  processing  at  the  U.  S.  Army  Engineer  Topo¬ 
graphic  Laboratories  (ETL)  is  to  develop  a  method  to  extract  mapping,  charting,  and 
geodetic  (MC&G)  data  efficiently  from  a  variety  of  digital  and  digitized  images.  Other 
objectives,  which  are  associated  with  this  basic  objective,  are  to  determine  (1)  what 
subset  of  the  MC&G  data  can  be  extracted  from  the  digital  images,  (2)  how  to  collect  the 
information,  and  (3)  whether  or  not  the  collection  process  can  be  automated  efficiently. 
To  accomplish  the  objectives,  methods  must  be  developed  to  register  similar  as  well  as 
dissimilar  images  to  one  another  to  determine  ground  coordinates  or  to  transfer  control. 
Methods  must  also  be  developed  to  use  basic  image  information,  such  as  texture,  to 
define  images  of  natural  and  cultural  features  found  on  digital  pictures.  The  purpose 
of  this  report  is  to  present  an  evaluation  of  conventional  image  matching  techniques  when 
matching  infrared  (IR)  imagery  to  panchromatic  (PANC)  imagery.  The  imagery  used  in 
this  exercise  will  also  be  used  in  a  texture  analysis.  The  texture  analyses  will  be  described 
in  other  ETL  research  notes. 


IMAGE  DATA 

The  images  used  in  this  analysis  were  exposed  on  28  October  1975  and  were  ob¬ 
tained  from  the  U.  S.  Geological  Survey  (USGS).  Two  aerial  cameras,  one  an  IR  and  the 
other  a  PANC,  were  exposed  simultaneously  and  in  such  a  way  that  the  IR  and  PANC 
images  almost  completely  overlapped  one  another.  A  brief  description  of  the  pertinent 
geometry  is  given  next  followed  by  a  description  of  the  image-scanning  process. 

Geometric  Description  ■  Five  pairs  of  IR  and  PANC  were  exposed  in  flight 
in  such  a  manner  that  the  base-height  ratio  between  successive  exposures  was  0.3.  Three 
scenes  that  appeared  on  all  five  IR  exposures  and  all  five  PANC  exposures  were  selected 
for  scanning.  The  pertinent  camera  parameters  associated  with  the  data  are  given  in 
table  1 . 
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TABLE  1.  CAMERA  DATA 


PANC 

IR 

Scale 

1  -.70000 

1:70000 

B/H 

0.3,  0.6,  0.9,  1.2 

0.3,  0.6,  0.9,  1.2 

f 

153.282  mm 

153.122  mm 

Spectral  Range 

0.4  -  0.7  pm 

0.7  -  0.9  pm 

The  five  IR  images  were  labeled  521,  through  561;  the  fire  PANC  images  were  labeled 
52P  through  56P.  Exterior  orientation  parameters  for  52P,  54P,  and  56P  were  provided 
by  USGS. 

Digitized  Data  ■  Three  scenes  that  appeared  on  all  10  (5  IR  images  and  5 
PANC  images)  exposures  were  selected  for  scanning.  The  upper  left  corner  of  each 
scene  was  marked  in  stereo  on  all  10  exposures  prior  to  scanning  on  the  PDS  1050A 
Automatic  Microdensitometer  system.  The  images  were  placed  in  the  microdensitometer 
comparator  so  that  the  scan  axis  was  nearly  parallel  to  the  base  line.  The  upper  left  corner 
of  each  scene  and  the  four  camera  fiducial  images  were  measured  before  and  after  the 
scanning  process.  These  data  are  used  to  calculate  a  transformation  from  pixel  space  to 
camera  space. 


The  pixel  spacing  and  the  line  spacing  was  14  pm  (micrometers).  The  pixel  diam¬ 
eter  was  20  pm;  1024  lines  and  1024  pixels  per  line  were  measured  for  each  of  the 
three  scenes  on  each  of  the  1 0  images.  The  microdensitometer  output  was  enhanced  to 
produce  8-bit  density  gray  shades  and  then  stored  on  disc  in  the  image-processing  system 
of  ETL.  The  three  scenes  (from  exposure  54)  are  shown  in  figures  1,  2,  and  3. 
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NUMERICAL  EXPERIMENT 


The  three  scenes  were  measured  on  the  IR  and  PANC  images  to  evaluate  the 
linear  correlation  coefficient  as  a  measure  of  similarity  when  matching  IR  to  PANC. 
The  basic  measure  of  effectiveness,  P,  is  defined  to  be  the  percentage  of  successful 
matches,  and  the  quantities  R,  C,  SP,  and  B/H  were  tested  for  their  worth  as  predictors 
of  a  successful  match.  The  criterion  for  a  successful  match  and  the  parameters  R,  C, 
SP,  and  B/H  are  defined  below. 

Match  Process  ■  A  rectangular  grid  of  points  was  established  on  each  scene 
of  541  .  The  upper  left  corner  on  each  scene  was  located  at  line  80  and  pixel  80;  29 
lines  and  29  pixels  per  line  were  specified  where  the  line  and  pixel  spacing  was  31.  Thus, 
29  x  29  =  841  points  were  defined  on  each  of  the  three  scenes  of  541,  and  the  match 
process  attempted  to  locate  the  corresponding  points  on  the  nine  remaining  digital 
images. 

A  match  point  was  estimated  on  the  dependent  image  and  then  refined  in  the 
following  manner.  A  5  x  9  array  of  correlations  values  was  developed  about  the  match 
point  estimate:  the  linear  correlation  coefficient  was  used  as  the  measure  of  similarity. 
The  line  and  pixel  location  of  the  correlation  maximum  was  computed  and  designated 
as  the  refined  match  point.  Note  that  the  correlation  function  was  computed  at  integer 
pixel  and  line  intersections;  whereas,  the  match  point  was  computed  to  fractions  of  line 
and  pixel  values.  Note  also  that  the  longer  dimension  of  the  correlation  function  was  in 
the  direction  of  major  parallax. 

The  largest  of  the  5  x  9  =  45  discrete  correlation  values  was  determined,  and  if 
that  value  was  anywhere  on  the  border  of  the  5x9  array  of  correlation  values,  the 
proces«  was  halted.  Next,  the  eight  correlation  values  surrounding  the  largest  of  the  dis¬ 
crete  values  and  the  largest  correlation  value  itself  were  input  to  a  routine  that  deter¬ 
mined  (ex,ey).  The  shifts,  (ex,ey),  define  the  location  of  the  maximum  of  the  correla¬ 
tion  function  with  respect  to  the  largest  of  the  discrete  values.  The  match  process  was 
halted  at  this  point  if  C  <  0,  1  ex  I  >  1,  or  |  ey  |  >  1.  The  parameter  C  is  the  product 
of  the  2-second  partial  derivatives  of  the  correlation  function  with  respect  to  x  and  y, 
respectively.  Both  of  these  values  must  be  negative  for  the  correlation  function  to  be 
concaved  downward. 
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The  following  values  were  output  for  each  successful  match: 

(Xp,Yp):  Match  coordinates 

R:  Correlation  value  at  the  match  point 

C:  Product  of  the  2-second  partial  derivatives  of  the 
correlation  function 

Sp:  Signal  power  of  match  point  on  the  independent  image. 

Thus  C  is  regarded  as  a  confidence  measure  in  that  large  values  indicate  a  narrow,  well- 
defined  correlation  function;  whereas,  small  values  indicate  a  flatter,  less  well-defined 
correlation  function.  Note  that  if  the  match  was  adjudged  unsuccessfully,  then  the  match 
point  estimate  was  output.  The  match  point  estimate  was  estimated  from  neighboring 
matches;  the  match  process  did  not  involve  image  shaping. 


The  following  values  were  output  at  the  completion  of  an  image  matching  run; 

R:  Average  Correlation  value 

C:  Average  C  value 

Sp:  Average  Signal  Power 
P:  Percentage  of  successful  matches 
The  quantities  R,  C,  and  Sp  were  averaged  over  successful  matches. 


A  large  number  of  matching  runs  were  performed  to  develop  statistical  variation 
in  the  output.  Ten  matching  runs  for  each  of  the  three  scenes  were  performed  when  1R 
was  matched  to  I  Rand  when  PANC  was  matched  to  PANC.  The  image  matching  was 
organized  according  to  base-height  (B/H)  ratio.  The  10  pairs  given  in  table  2  and  the  16 
pairs  in  table  3  were  replicated  by  using  two  match  window  sizes,  namely  21  by  21 
and  31  by  31. 


TABLE  2. 


MATCH  PAIRS  FOR  IR  AND  FOR  PANC 


B/H 

IMAGE  PAIRS 

0.3 

54-55 

54-53 

53-52 

0.6 

54-56 

54-52 

53-55 

0.9 

52-55 

53-56 

1.2 

52-56 

TABLE  3.  MATCH  PAIRS  FOR  MATCHING  IR  TO  PANC 


B/H 


0.0 

52-52 

53-53 

54-54 

55-55 

0.3 

54-55 

54-53 

53-52 

55-56 

0.6 

54-56 

54-52 

53-55 

0.9 

52-55 

53-56 

1.2 

52-56 

56-52 

Note  that  16  matching  runs  for  each  of  the  three  scenes  were  made  when  IR  was  matched 
to  PANC.  In  this  case,  an  extra  match  run  was  possible  when  B/H  =  1.2;  also,  it  was 
possible  to  acquire  five  matching  runs  for  B/H  =  0.9. 


Numerical  Results  ■  The  primary  objective  of  this  work  effort  was  to  eval¬ 
uate  the  process  of  determining  'corresponding  points  between  IR  and  PANC  using 
conventional  correlation  methods.  The  match  windows  were  not  shaped  to  reflect  local 
parallax,  and  no  strategies  were  employed  under  difficult  conditions.  If  the  process  failed 
in  any  of  the  ways  described  previously,  than  the  match  was  regarded  as  a  failure.  The 
same  procedure  was  used  when  IR  was  matched  to  IR  and  when  PANC  was  matched  to 
PANC.  The  purpose  of  the  latter  exercises  was  to  provide  a  numerical  comparison  for 
the  primary  objective.  Tables  4  through  9  are  averaged  values  of  the  results.  For  example, 

P  pertains  to  the  percentage  of  successes  averaged  over  the  appropriate  number  of  image 
pairs. 


TABLE  4.  MATCH  RESULTS  FOR  PANC  TO  PANC  -21x21 


SCENES 


B/H  =  0.3  A  B  C 


p 

34.10 

47.00 

72.29 

R 

0.366 

0.433 

0.525 

C 

0.0051 

0.0080 

0.0085 

Sp 

591 

679 

974 

B/H  =  0.6 

P 

27.78 

34.56 

38.05 

R 

0.312 

0.351 

0.411 

C 

0.0047 

0.0055 

0.0051 

Sp 

520 

580 

943 

B/H  =  0.9 

P 

30.20 

35.61 

32.19 

R 

0.309 

0.297 

0.315 

C 

0.0044 

0.0046 

0.0037 

Sp 

558 

549 

817 

B/H  =  1.2 

P 

30.44 

33.18 

21.69 

R 

0.314 

0.278 

0.238 

C 

0.0045 

0.0049 

0.0036 

§P 

569 

493 

783 
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TABLE  5.  MATCH  RESULTS  FOR  PANCTO  PANC  -  31x31 


SCENES 


B/H  =  0.3  A  B 


p 

34.04 

63.11 

78.98 

R 

0.399 

0.487 

0.543 

C 

0.0027 

0.0054 

0.0056 

Sp 

654 

747 

1059 

B/H  =  0.6 

P 

27.35 

35.55 

36.51 

R 

0.357 

0.365 

0.420 

C 

0.0025 

0.0029 

0.0028 

Sp 

618 

607 

996 

B/H  =  0.9 

P 

30.14 

31.81 

31.18 

R 

0.318 

0.312 

0.344 

C 

0.0020 

0.0023 

0.0019 

Sp 

619 

588 

942 

B/H  =  1.2 

P 

29.01 

31.39 

1931 

R 

0.338 

0.282 

0.268 

C 

0.0019 

0.0020 

0.0018 

Sp 

682 

505 

764 

1  3 
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TABLE  6.  MATCH  RESULTS  FOR  IR  TO  IR  -  2]  x  21 

SCENES 


B/H  =  0.3 

A 

B 

C 

P 

64.06 

67.87 

74.87 

R 

0.526 

0.606 

0.609 

C 

0.0063 

0.0149 

0.0115 

Sp 

533 

785 

884 

B/H  =  0.6 

P 

50.89 

43.00 

42.02 

R 

0.473 

0.484 

0.459 

C 

0.0049 

0.0093 

0.0064 

Sp 

513 

785 

898 

B/H  =  0.9 

P 

41.14 

37.16 

28.63 

R 

0.439 

0.401 

0.350 

C 

0.0042 

0.0059 

0.0042 

Sp 

486 

646 

647 

B/H  =  1.2 

P 

27.47 

33.18 

24.01 

R 

0.350 

0.330 

0.289 

C 

0.0039 

0.0046 

0.0035 

Sp 

471 

514 

672 

14 


TABLE  7.  MATCH  RESULTS  FOR  IR  TO  IR  -  31x31 

SCENES 


B/H  =  0.3  A  B  C 


p 

76.90 

70.88 

79.61 

R 

0.573 

0.637 

0.618 

C 

0.0037 

0.0112 

0.0088 

Sp 

622 

912 

972 

B/H  =  0.6 

P 

55.65 

43.36 

44.19 

R 

0.522 

0.522 

0.476 

C 

0.0026 

0.0064 

0.0042 

Sp 

586 

883 

966 

B/H  =  9.0 

P 

37.10 

37.02 

25.45 

R 

0.487 

0.429 

0.363 

C 

0.0020 

0.0035 

0.0023 

Sp 

621 

724 

874 

B/H  =  1.2 

P 

42.09 

31.87 

25.99 

R 

0.427 

0.349 

0.282 

C 

0.0020 

0.0025 

0.0017 

Sp 

568 

681 

744 

TABLE  8.  MATCH  RESULTS  FOR  IR  TO  PANC  -  21  x  21 


SCENES 


B/H  =  0.0 

A 

B 

C 

P 

33.29 

53.03 

64.54 

R 

0.273 

0.436 

0.494 

C 

0.0043 

0.0102 

0.0102 

Sp 

459 

674 

895 

B/H  =  0.3 

P 

38.08 

42.79 

52.53 

R 

0.316 

0.375 

0.439 

C 

0.0046 

0.0083 

0.0071 

Sp 

498 

627 

973 

B/H  =  0.6 

P 

32.78 

33.69 

33.32 

R 

0.278 

0.312 

0.345 

C 

0.0041 

0.0063 

0.0046 

Sp 

482 

698 

874 

B/H  =  0.9 

P 

35.86 

31.51 

27.74 

R 

0.262 

0.275 

0.277 

C 

0.0042 

0.0047 

0.0038 

Sp 

449 

507 

810 

B/H  =  1.2 

P 

26.93 

28.72 

25.50 

R 

0  2  i  0 

0.232 

0.239 

C 

0.0039 

0.0040 

0.0036 

Sp 

432 

575 

778 

I  6 


TABLE  9.  MATCH  RESULTS  FOR  IR  TO  PANC  -  31x31 


SCENES 


B/H  =  0.0 

A 

B 

C 

P 

45.49 

51.74 

72.77 

R 

0.368 

0.446 

0.496 

C 

0.0053 

0.0079 

0.0080 

Sp 

606 

835 

945 

B/H  =  0.3 

P 

42.98 

37.03 

53.31 

R 

0.353 

0.362 

0.418 

C 

0.0059 

0.0057 

0.0052 

Sp 

618 

818 

1032 

B/H  =  0.6 

P 

32.50 

32.62 

34.28 

R 

0.265 

0.314 

0.340 

C 

0.0018 

0.0037 

0.0027 

Sp 

563 

803 

969 

B/H  =  0.9 

P 

28.35 

30.14 

25.99 

R 

0.259 

0.267 

0.257 

C 

0.0014 

0.0023 

0.0018 

Sp 

547 

637 

844 

B/H  =  1.2 

P 

28.42 

24.55 

24.64 

R 

0.249 

0.211 

0.208 

C 

0.0015 

0.0018 

0.0014 

Sp 

508 

607 

872 
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STATISTICAL  ANALYSIS 


Two  statistical  analyses  were  performed  on  the  data.  The  first  was  an  analysis 
of  gray  shade  and  signal  power  variation  to  determine  if  an  a  priori  evaluation  of  signal 
power  or  gray  shade  could  be  used  to  identify  those  scenes  most  likely  to  produce 
good  matches  from  those  unlikely  to  produce  good  matches.  The  second  analysis 
was  a  multiple  regression  analysis  to  determine  which  of  the  match  parameters  and 
which  of  the  geometric  parameters  were  significant  in  determing  a  successful  match. 


Gray  Shade  and  Signal  Power  Variation  ■  Each  of  the  30  digitized  images 
(3  scenes  on  5  IR  and  5  PANC  exposures)  was  segmented  into  16  blocks  where  each 
block  was  dimensioned  256  by  256.  The  following  five  quantities  were  computed  for 
each  of  the  480  blocks. 

gm  a  x :  Largest  gray  shade 

8m  in :  Smallest  gray  shade 

g  :  Average  gray  shade 

SP  :  Signal  Power  or  block  variance 

Ho  :  Entropy 

Several  statistical  measures  were  computed  within  images  and  over  images.  It  was 
assumed  that  the  (  l,k  )  block  (  1  =  1,4  and  k  =  1,4  ),  taken  from  a  specific  scene, 
corresponded  to  the  (  l,k  )  block  taken  from  the  same  scene  extracted  from  a  differ¬ 
ent  exposure. 


The  objective  was  to  compute  a  variety  of  scene  statistics  and  then  to  note 
whether  a  relationship  between  the  scene  statistics  and  successful  matches  could  be 
determined.  This  portion  of  the  experiment  produced  little  or  no  predictive  infor¬ 
mation.  However,  it  was  noted  that  the  ratio  of  the  scene  signal  power  to  a  within- 
estimate  signal  power  showed  a  reasonable  amount  of  consistency  when  compared 
to  the  percentage  of  successful  matches.  The  results  are  summarized  in  table  10. 


TABLE  10. 

SCENE  STATISTICS 

SENSOR 

SCENE 

°i\2 

°z22 

°zl2/°z22 

A 

1961 

1432 

1.37 

IR 

B 

2290 

1735 

1.32 

C 

1584 

1152 

1.38 

A 

1791 

1541 

1.16 

PANC 

B 

2010 

1123 

1.79 

C 

2368 

1747 

1.36 

The  scene  signal  power,  ozl2,  and  the  within-estimate  signal  power,  az22,  were 
computed  in  each  of  the  six  cases  by  averaging  the  five  exposures.  The  first  three  and 
the  last  ratios  are  similar  in  value  (azl  2/az22  —  1.4)  and,  in  fact,  correspond  to  the 
more  successful  stereo  matching  situations.  The  ratio  for  PANC  Scene  A  was  the  smallest 
(azl2/ozi2  =  1.16),  and  the  corresponding  exposures  produced  the  poorest  matches. 
The  ratio  for  PANC  Scene  B  was  the  largest  ( ozl2/uz2 2  =  1.79),  and  the  corresponding 
exposures  produced  the  next  worst  match  results.  These  results  indicate  that  a  moderate 
amount  of  signal  power  variation  over  a  scene  is  more  likely  to  produce  better  match 
results  than  either  a  small  or  a  large  amount.  It  is  unlikely  that  this  measure  of  success¬ 
ful  match  can  be  put  to  use  in  an  automatic  process. 


Multiple  Regression  Analysis  •  The  objective  was  to  determine  which,  if 
any,  of  the  four  parameters  R,  C,  SP  and  B/H  could  be  used  to  predict  whether  or  not 
a  particular  match  was  successful.  The  parameters  R  and  C  are  functions  of  the  imagery 
and  of  the  match  process,  SP  is  a  function  of  the  imagery,  and  B/H  is  a  function  of  the 
exposure  geometry.  A  multiple  regression  analysis  of  the  four  parameters  on  the  number 
of  successes  was  used  as  a  means  for  making  the  determination. 


In  the  linear  regression  equation  given  in  the  appendix,  it  was  assumed  at  the  outset 
that  the  coefficients  were  zero.  If  any  of  the  90  percent  intervals  about  the  coefficient 
extimates  included  zero,  then  the  hypothesis  of  no  significance  would  be  supported  at 
the  0.10  confidence  level.  For  example,  the  hypothesis  that  the  coefficient  of  R  is  zero 
was  rejected  in  five  of  the  six  analyses.  The  data  of  the  IR  to  IR  (31x31)  supported  the 
hypothesis  of  no  significance  in  R.  The  hypothesis  that  the  coefficient  of  B/H  is  zero  was 
supported  by  all  the  tests.  The  hypothesis  that  the  coefficient  of  C  is  zero  was  supported 
by  all  but  the  IR  to  PANC  (21  x  21)  test.  The  hypothesis  that  the  coefficient  of  SP  is 
zero  was  rejected  by  the  IR  to  PANC  (21  x  21)  test  and  by  the  IR  to  PANC  (31  x  31) 
test. 


IP 


. tm  ■■■■■■pi . ■ 


The  results  of  the  tests  indicate  that  the  linear  correlation  coefficient  R  should 
be  a  good  predictor  of  a  successful  match.  The  signal  power  SP,  associated  with  the  match 
window,  is  a  fair  predictor,  especially  when  1R  is  matched  to  PANC.  The  confidence 
value  C  turned  out  to  be  of  marginal  value  for  predicting  successful  matches,  and  the 
base-height  (B/H)  ratio  turned  out  to  be  of  no  value. 


DISCUSSION 

The  conclusion  drawn  from  the  regression  analysis  that  the  base-height  (B/H) 
ratio  was  of  no  value  in  predicting  a  successful  match  is  contrary  to  experience  and  also 
contrary  to  previous  studies.1  In  fact,  the  six  correlation  matrices  derived  from  this 
study  (see  appendix)  show  that  B/H  is  negatively  correlated  with  the  three  variates  P 
(percentage  of  successes),  R  (correlation  value  of  match),  and  C  (confidence  measure). 
Note  that  the  correlation  value  R  is  a  measure  of  similarity  between  two  points  on 
separate  images;  whereas,  the  correlation  matrices  given  in  the  appendices  describe  the 
linear  dependence  of  the  variates  P,  R,  C,  SP  and  B/H  upon  one  another.  The  negative 
correlation  values  imply  that  as  B/H  increases,  the  values  of  P,  R,  and  C  decrease.  That 
the  regression  analyses  did  not  substantiate  this  observation  can  be  explained  by  re¬ 
viewing  the  percentage  of  successes  as  a  function  of  B/H.  Consider  tables  1 1  and  1 2 
extracted  from  tables  4  through  9. 


*M.  Crombie  and  L.  Gambino,  “Digital  Stereo  Ph  otogram  m  etry  ”,  Presented  to  Congress 
of  the  International  Federation  of  Surveyors  (FIG).  Comm  ission  V,  Stockholm,  Sweden, 
June  1977. 


20 


TABLE  11.  PERCENTAGE  OF  SUCCESSES  (21  x  21  WINDOW) 


PANC-PANC 


B/H 

A 

B 

C 

0.0 

0.3 

34 

47 

72 

0.6 

28 

35 

38 

0.9 

30 

36 

32 

1.2 

30 

33 

32 

TABLE  12.  PERCENTAGE 


PANC-PANC 


B/H 

A 

B 

C 

0.0 

0.3 

34 

63 

79 

0.6 

27 

36 

37 

0.9 

30 

32 

31 

1.2 

29 

31 

19 

IR-1R  IR-PANC 


A 

B 

C 

A 

B 

C 

33 

53 

65 

64 

68 

75 

38 

43 

53 

51 

43 

42 

33 

34 

33 

41 

37 

29 

36 

32 

28 

27 

33 

24 

27 

29 

26 

SUCCESSES  (31x31  WINDOW) 
IR-1R  IR-PANC 


A 

B 

C 

A 

B 

C 

45 

52 

73 

77 

71 

80 

43 

37 

53 

56 

43 

44 

33 

33 

34 

37 

37 

25 

28 

30 

26 

42 

32 

26 

00 

rj 

25 

25 

There  is  a  downward  trend  in  P  as  B/H  increases;  however,  the  slope  decreases 
to  zero  as  B/H  >  0.6.  In  fact,  the  P  values  for  scene  A  (PANC  to  PANC  and  IR  to 
PANC)  have  little  or  no  slope  over  the  entire  range  of  B/H.  These  observations,  along 
with  the  limited  number  of  data  points  plus  the  other  variables  explaining  the  percentage 
of  success  P,  mean  that  the  coefficient  of  B/H  in  the  regression  equation  being  zero  is 
not  an  unlikely  occurrence.  The  single  variable  R  explained  most  of  the  variation  in  P. 
Consider  the  six  partial  correlation  matrices  in  the  appendix.  The  partial  correlation 
matrix  describes  the  linear  dependence  among  the  variates  P,  R,  and  C  when  the  signal 
power  (SP)  and  the  base-height  (B/H)  ratio  are  constant.  The  partial  correlation  between 
P  and  R  is  0.923  <  pPR  •  SP,  B/H  <  0.953. 


There  is  a  very  noticable  decrease  in  the  percentage  of  successes  as  B/H  >  0.3 
for  scene  B,  and  especially  for  scene  C.  The  relatively  large  signal  power  of  scene  C,  and 
to  some  extene  that  of  scene  B,  implies  an  abundance  of  edges  that  become  obscured 
when  viewed  from  different  aspects.  Thus,  two  exposures  of  a  busy  ground  scene  will 
be  noisy  with  respect  to  one  another,  or  common  ground  detail  is  obscured  owing  to 
different  perspectives.  Also  trying  to  cancel  noise  by  more  measurements  (enlarging 
the  matching  windows)  did  not  help.  In  fact,  the  number  of  successes  decreased  as  the 
window  size  was  increased  from  21  to  31,  when  B/H  >  0.3. 


However,  there  are  at  least  two  ways  to  improve  the  match  process.  The  first  is 
to  shape  the  image  to  account  for  terrain  elevation  changes  and  to  account  for  exposure 
geometry  differences.  The  second  is  to  use  only  the  common  information,  especially  if 
the  images  are  dissimilar  such  as  when  matching  IR  to  PANC.  Consider  table  13,  which 
shows  the  relative  scale  change  of  a  ramp  tilted  a  degree  from  the  datum  when  viewed 
from  two  vertical  exposures.2  The  ramp-like  object  is  assumed  to  be  midway  between 
the  two  exposure  stations.  It  can  be  seen  from  the  table  that  for  small  values  of  a  and 
B/H,  the  scale  change  may  not  be  important.  However,  as  either  parameter  increases, 
the  relative  scale  increases  nonlinearly.  This  means  that  unless  one  image  is  resampled 
to  reflect  the  scale  change,  the  matching  windows,  even  when  centered  over  corres¬ 
ponding  points,  will  not  necessarily  indicate  a  match.  The  beneficial  effects  of  image 
shaping,  especially  in  steep  areas,  was  demonstrated  in  another  research  note.3 


2M.  Crombie  and  L.  Gambine,  “Digital  Stereo  Photogram  m  etry,”  Presented  to 
Congress  of  the  In  ternational  Federation  of  Surveyors  (FIG ).  Com  m  issio  n  V  ,  Stockholm, 
Sweden,  June  1977. 

3M.  Crombie,  Stereo  Analysis  of  A  Specific  Digital  Model  Sample  From  A  erial  Imagery , 
U.  S.  Army  Engineer  Topographic  Laboratories,  Tort  Belvoir,  VA  ETL--0072, 
September  1  976,  AD-A033  567. 
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TABLE  13.  IMAGE  SCALE  CHANGE 


B/H 

0° 

5° 

10° 

a 

15° 

O 

o 

25° 

O 

O 

0.4 

1.00 

1.04 

1.07 

1.11 

1.16 

1.21 

1.26 

0.6 

1.00 

1.05 

1.11 

1.17 

1.25 

1.33 

1.42 

0.8 

1.00 

1.07 

1.15 

1.24 

1.34 

1.46 

1.60 

1.0 

1.00 

1.09 

1.19 

1.31 

1.44 

1.61 

1.81 

That  the  exposures  1R  and  PANC  are  dissimilar  can  be  demonstrated  by  a  visual 
examination  of  the  exposure  pairs  for  scenes  A,  B, and  C.  Note  that  the  exposure  pairs 
are  of  nearly  the  same  area,  and  they  were  exposed  from  nearly  identical  locations  with 
little  or  no  relative  tilts.  A  numerical  verification  of  the  same  fact  is  seen  in  tables  1 1  and 
12,  where,  when  B/H  =  0,  the  percentages  of  successful  matches  varies  from  33  to  73 
percent  instead  of  100  percent. 


The  high  frequency  information  appears  to  contribute  more  to  the  match  success, 
when  B/H  <  0.6,  than  the  low  frequency  information.  This  can  be  shown  by  noting 
the  sharp  increase  in  success  for  scene  B,  and  especially  for  scene  C,  as  B/H  <  0.6. 
Both  of  these  scenes  have  relatively  high  signal  power;  whereas,  the  low  signal  power 
of  scene  A  produces  little  change  in  success  as  B/H  varies.  It  can  be  shown  that  high 
frequency  information  in  an  image  scene  results  in  a  shorter  pull  in  range.4  This  is 
equivalent  to  a  more  spike-like  correlation  function  or  equivalently  to  a  large  C  value. 
In  tables  4  through  9,  scenes  B  and  C  have  much  larger  C  values  than  scene  A  for 
B/H  <  0.6. 


4  M  .  Crombic  and  R.  Rand,  An  Evaluation  of  the  M  e  thod  of  D  e  term  inin  g  Parallax  From 
Measured  Phase  Difference,  U.  S.  Army  Unginccr  Topographic  Laboratories,  Fort 
Bclvoir,  VA,  F.TL-0145,  December  1  977,  AD-A056  006. 


The  percentage  of  successes  given  in  tables  1 1  and  1 2  are  conservative.  The  success 
criteria  was  arbitrarily  imposed,  and  no  strategies  were  used  to  increase  the  number  of 
successes.  One  of  the  best  ways  to  increase  the  number  of  successful  matches  is  to  per¬ 
form  a  preprocessing  operation  for  each  match  attempt  to  remove  all  adverse  elfects 
of  geometric  distortion  and  to  accentuate  image  similarities.  For  similar  imagery  (IR 
to  IR  and  PANC  to  PANC),  this  would  involve  image  shaping  and  filtering  out  in¬ 
creasing  amounts  of  high  frequency  data  on  B/H  increases.  For  dissimilar  imagery 
(IR  to  PANC),  this  would  also  involve  image  shaping;  however,  the  high  frequency 
information  should  be  enhanced  so  that  the  common  image  structure  can  be  registered. 
Corresponding  pixels  on  IR  and  PANC,  in  most  cases,  will  have  entirely  different 
illumination  values,  even  when  B/H  =  0;  whereas,  image  structure  (roads,  fields,  build¬ 
ings,  etc.)  appear  to  be  immune  to  sensor  changes.  This  is  not  entirely  true;  for  example, 
note  the  creek  found  on  scene  B  of  the  IR,  but  hardly  noticeable  on  the  PANC. 


It  is  expected  that  a  different  algorithm  than  the  conventional  algorithm  used 
in  this  study  should  be  used  to  develop  match  points  between  IR  and  PANC  images. 
Since  it  has  been  asserted  here  that  the  principal  commonality  between  the  two  record 
types  is  image  structure,  then  a  reasonable  approach  would  be  to  exploit  the  similarities 
between  the  density  gradient  functions  of  the  image  pairs.5 


There  are  at  least  five  parameters  that  can  be  used  to  evaluate  a  match  when  the 
conventional  area  correlation  method  is  used. 

1 .  R:  Correlation  value  at  the  match  point. 

2.  C;  Product  of  the  two  partial  derivatives  of  the  Correlation  function. 

3.  SP:  Signal  Power  in  the  neighborhood  of  the  match  point. 

4.  Sx :  Scale  difference  between  corresponding  image  segments. 

5.  Sx  :  Computed  shift  in  X-parallax. 


5Richard  H.  Hudgin,  Image  Matching  Using  Structure  In  form  a  tion,  SPIE ,  Vol.  117, 
1  977,  pp.  1 26-131. 


Parameters  R  and  C  are  a  tunction  of  the  correlation  process,  i.e.  resampling, 
window  size,  etc.  The  parameter  SP  is  a  function  of  the  ground  detail  and  of  the  sensor; 
whereas,  Sx  is  dependent  upon  the  terrain  and  the  exposure  geometry.  The  parameter 
6X,  which  is  a  model  error,  reflects  unexpected  change  in  terrain  slopes,  loss  or  partial 
loss  ot  correlation,  recovery  of  correlation  owing  to  previous  errors,  etc.  Since  the  match 
operation  involves  a  match  point  estimation  followed  by  a  refinement  process,  small 
values  ot  6X  imply  a  successtul  match;  whereas,  larger  values  imply  that  something 
may  be  amiss.  A  series  ot  large  values  of  5X ,  especially  when  the  other  indicators  are 
suspect,  implies  that  the  process  is  out  of  control.  If  Sx,  the  scale  factor,  strays  too  far 
from  unity,  then  the  match  process  must  work  with  pixel  values  diluted  in  resolution. 
Therefore,  Sx  values  that  are  significantly  different  from  one  indicate  match  difficulties. 
The  parameter  C  was  used  in  one  study,6  and  the  three  parameters  R,  C,  and  S 
were  used  in  another  study.7  All  five  parameters  were  used  in  a  final  study.8  The  three 
study  results  pertain  to  aerial  imagery  exposed  in  a  panchromatic  camera.  It  was  shown 
in  this  report  that  of  the  three  parameters  R,  C,  and  SP,  the  correlation  value,  R  was 
the  most  useful  in  predicting  a  valid  match.  The  signal  power,  SP,  turned  out  to ’be  a 
fair  predictor,  especially  when  matching  IR  to  PANf.  The  parameter  C  turned  out 
to  be  of  marginal  value. 


P.  Rosenberg,  K.  Erickson  and  G.  Rowe,  Digital  Mapping  System 
Processing,  U.  S.  Army  Engineer  Topographic  Laboratories.  Fort 
ETL-CR  -74-6,  M  ay  1  9  74  ,  A  D-7  8  2  2  30. 
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CONCLUSIONS 


1 .  The  correlation  value,  R,  is  the  best  indicator  of  a  successful  match. 

2.  The  signal  power,  SP,  turned  out  to  be  a  fair  indicator  of  a  successful  match, 
especially  when  matching  IR  imagery  to  PANC  imagery. 

3.  The  product  of  the  two  partial  derivatives  of  the  correlation  function,  C,  turned 
out  to  be  of  marginal  use. 

4.  Match  successes  decrease  rapidily  as  the  base-height  (B/H)  ratio  increases;  this 
is  especially  true  for  scenes  with  large  signal  power. 

5.  Image  structure  should  be  exploited  when  registering  IR  imagery  to  PANC 
imagery. 

6.  Algorithms  other  than  the  conventional  area  correlation  procedure  must  be  de¬ 
veloped  and  tested  for  the  successful  registration  of  IR  imagery  to  PANC  imagery. 
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APPENDIX.  Multiple  Regression  Results 


The  data  given  in  this  appendix  are  results  from  six  regression  analyses.  The  six 
sets  pertain  to  the  three  match  situations  IR  to  IR,  PANC  to  PANC,  and  IR  to 
PANC,  and  to  the  two  window  sizes  used  in  the  match  process.  The  regression  equation 
in  all  cases  was  the  following: 

P  =  /i  +  axR  +  bxC  +  cxSP  +  dx  B/H  +  e 

where 

P:  Percentage  of  successful  matches 

li:  P-[axR  +  bxC  +  cxSP  +  dx  B/H] 

R:  Average  correlation  of  the  successful  matches 
C:  Average  confidence  value  of  the  successful  matches 
SP:  Average  signal  power  of  the  successful  matches 
B/H:  Base-height  ratio 

e:  Model  Error.  The  expected  value  of  e  is  zero  and  the  variance  of 
e  is  o2 . 

There  were  30  observation  equations  for  each  of  the  IR  to  IR  matches  and  for  each 
of  the  PANC  to  PANC  matches.  There  were  48  observation  equations  for  each  of  the 
IR  to  PANC  matches. 


The  regression  analysis  results  were  computed  and  the  F-statistic  was  used  to 
determine  whether  or  not  the  regression  coefficients  a,  b,  c,  and  d  were  significantly 
different  from  zero.9  The  initial  hypothesis  was  that  each  coefficient  was  zero,  i.e.  the 
corresponding  parameters  were  not  linearly  related  to  match  success. 


^Donald  F.  Morrison,  Multivariate  Statistical  Methods,  McGraw-Hill  Book  Company, 
New  York,  1976,  pp.  107. 
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There  are  two  correlation  matrices  associated  with  each  of  the  six  sets  of  results; 
each  are  presented  in  upper  triangular  form.  The  first  is  the  5  by  5  correlation  matrix 
of  the  input  data,  and  the  second  is  the  3  by  3  partial  correlation  matrix,  wherein  signal 
power  (SP)  and  base-height  (B/H)  are  held  constant.  The  next  set  of  data  includes 
the  regression  coefficients  and  their  90  percent  confidence  bounds.  The  last  two  values 
in  each  of  the  six  sets  is  the  square  of  the  multiple  correlation  coefficient  and  the  stan¬ 
dard  error  of  estimate. 


IR  to  IR  (21  x  21) 


Correlation  Matrix 


r  1.000 


0.924 

1.000 


0.639 

0.336 

-0.854 

0.807 

0.476 

-0.913 

1.000 

0.650 

-0.640 

1.000 

-0.388 

1.000 

Partial  Correlation  Matrix 


^1.000 


0.953 

1.000 


0.716 

0.843 

1.000 


Regression  Coefficients  and  90%  bounds 


H  =  -57.4 
a  =  242  ±  127 
b  =  -1361  ±  1828 
c  =  -0.0038  ±  0.0270 
d  =  7.73  ±  52.1 


Multiple  Regression  Coefficient  Squared 
P2  =  0.889 


Standard  Error  of  Estimate 
a  =  6.3 
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I 


PANC  to  PANC  (21  x  21) 


Correlation  Matrix 

(1.000  0.896  0.852  0.599  -0.499 

1.000  0.841  0.612  -0.593 

1.000  0.426  -0.596 

1.000  -0.106 

i  1.000 


M  =  -30.5 
a  =  99  ±  93 
b  =  3933  ±  4191 
c  =  .0124  ±  0.0281 
d  =  5.7  ±  16.0 


Multiple  Regression  Coefficient  Squared 
P2  =  0.854 

Standard  Error  of  Estimate 
o  =  6.9 


29 


i 


1R  to  PANC  (21  x  21) 


Regression  Coefficients  and  90%  bounds 

pi  =  1.5 
a  =  80  ±  54 
b  =  1905  ±  1595 
c  =  0.0025  ±  0.0021 
d  =  -2.54  ±  8.79 


Multiple  Regression  Coefficient  Squared 
P2  =  0.835 

Standard  Error  of  Estimate 
a  =  6.5 
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IR  to  1R  (31  x  31) 


Correlation  Matrix 


(1.000  0.867  0.568  0.130 

1.000  0.726  0.241 

1.000  0.650 

1.000 


Partial  Correlation  matrix 


0.923 

1.000 


0.614 

0.740 

1.000 


Regression  Coefficients  and  90%  bounds 

H  =  57.7 
a  =  69  ±  191 
b  =  597  ±  3477 
c  =  -0.0251  ±  0.0577 
d  =  -36.5  ±  49.4 


Multiple  Regression  Coefficient  Squared 
P2  =  0.799 


Standard  Error  of  Estimate 
a  =  9.7 


3  I 


-0.859\ 
-0.912  \ 
-0.635  I 
-0.330  I 
1.000/ 


PANC  to  PANC  (31x31) 


Correlation  Matrix 


(1 .000  0.910 

1.000 


0.879 

0.541 

-0.610 

0.876 

0.632 

-0.784 

1.000 

0.531 

-0.655 

1.000 

-0.412 

1.000 

Partial  Correlation  Matrix 


0.937 

1.000 


0.918 

0.920 

1.000 


Regression  Coefficients  and  90%  bounds 

M  =  -59.2 
a  =  21 1  ±  133 
b  =  3948  ±  5472 
c  =  -0.0074  ±  0.0016 
d  =  17.2  ±  21.8 


Multiple  Regression  Coefficient  Squared 
P2  =  0.884 


Standard  Error  of  Estimate 
a  =  7.3 
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IR  to  PANC  (31x31) 


Correlation  Matrix 

-0.624\ 
-0.709  1 
-0.592  1 
-0.272  I 
1.000/ 


Partial  Correlation  Matrix 

/  1.000  0.929  0.748 

|  1.000  0.785 

\  1 .000 

Regression  Coefficients 

H  =  -13.4 
a  =  151  ±  67 

b  =  331  ±  1527 

c  =  0.0037  ±  0.0026 
d  =  0.75  ±  13.3 


Multiple  Regression  Coefficient  Squared 
P2  =  0.794 


Standard  Error  of  Estimate 
a  =  8.8 


1.000 


0.889 

1.000 


0.640 

0.536 

0.686 

0.574 

1.000 

0.385 

1.000 

3  3 


